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11) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)Q Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2.D Certified copies of the priority documents have been received in Application No. . 



3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 



1) I22 Notice of References Cited (PTO-892) 

2) LZ| Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) ^ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 

Paper No(s)/Mail Date lOf^ f <H .J 



4) CI Interview Summary (PTO-413) 

Paper No(s)/Mail Date. . 

5) □ Notice of Informal Patent Application (PTO-152) 
. 6) □ Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 1-04) 



Office Action Summary 



Part of Paper No./Mail Date 20052005 



Application/Control Number: 09/891,030 Page 2 

Art Unit: 2654 

DETAILED ACTION 
Claim Rejections - 35 USC § 102 

1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

2. Claims 1, 2, 4-6, 10-16, 18-20, and 24-28 are rejected under 35 U.S.C. 102(b)as 
being anticipated by Sakiyama et_aL (5,659,764). 

As to claim 1 , Sakiyama et al. teach: 

a speech input responsive to verbal communication, (a voice interface unit for 
converting a signal sent from the microphone into a kana and kanji coexistent sentence 
recognizable by a computer, col. 8, lines 24-26); 

a speech recognition processor responsive to said speech input and creating an 
electronic output representing said verbal communication, (a voice recognition 
apparatus for creating a textual output from a voice input, col. 8, lines 26-28); 

a database storing words and non-textual graphic image designators 
corresponding to said words, (signlanguage wordCG pattern storage, allowing the 
input to be matched to sign language-like expression, col. 8, lines 38-41 , and line 48); 
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a processor responsive to said electronic output representing said verbal 
communication to access a graphic image designator from said database which 
represents said verbal communication, (a spoken language processing unit for 
performing a spoken language analysis of the inputted voice language composition, 
rearranging sign language words so that they maymeet a sign language-like expression, 
col. 8, lines 37-41); and 

a graphic image generator responsive to said graphic image designator to 
generate a graphic image, which represents said verbal communication, (CG animation 
generation unit for supplying a graphic, matching the voice input, col. 8, lines 38-41 , and 
46-48). 

As to claim 15, Sakiyama et al. teach: 

a text source for generating electronic output representing words (a kana and 
kanji coexistent sentence recognizable on a computer, realizable by a voice recognition 
apparatus, col. 8, lines 27-29); 

a database storing words and non-textual graphic image designators 
corresponding to said words (a CG pattern storage in accordance with an arrangement 
of sign language words, with the sign language animations, col. 8, lines 47-50), 

a processor responsive to said electronic output representing words to a access 
a graphic image designator from said database which represents said words (a CG 
animation generation unit applies a sign language animation to the corresponding sign 
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language word denoted by the spoken language processing unit, col. 8, lines 46-48); 
and 

a graphic image generator responsive to said graphic image designator to 
generate a graphic image which represents said words (a CG animation generation unit 
applies a sign language animation to the corresponding sign language word denoted by 
the spoken language processing unit, col. 8, lines 46-48). 

As to claims 2 and 16, Sakiyama et al. teach at least one computing device, 
which includes said speech processor (computer with a voice recognizer, col. 8, lines 
28-30) 

As to claims 4 and 18, Sakiyama et al. teach the computing device comprises a 
display screen for displaying said graphic image (display screen, col. 8, lines 54-55). 

As to claims 5 and 1 9, Sakiyama et al. teach one communication device 
accessing a network (Fig. 22, depicts two computer devices linked together for 
communication, creating a network) 

As to claims 6 and 20, Sakiyama et al. teach the speech recognition processor 
comprises an acoustic processor and a word decoder, (a voice interface unit for 
converting a voice input signal from a microphone into a kana and kanji coexistent 
sentence recognizable by a computer, col. 8, lines 25-28). 
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As to claims 10 and 24, Sakiyama et al. teach: 

inputting verbal communication to a processor (a microphone for inputting words, 
col. 8, lines 24-26); 

matching, in said processor, said verbal communication with graphic, non-textual 
images representing said verbal communication (matching the verbal communication 
with an animation representing a sign language, col. 8, lines 38-41 , and lines 48-50); 
and 

outputting from said processor said graphic images (displaying the animations on 
a display device, col. 8, lines 52-55). 

As to claims 1 1 , and 25, Sakiyama et al. teach transmitting said graphic images 
to a display screen (displaying the animations on a display device, col. 8, lines 52-55). 

As to claims 12 and 26, Sakiyama et al. do not teach responding to graphic 
images by inputting additional verbal communication (Fig. 22 depicts a communication 
network, where output is to both users through monitors, (elements 7 and 28), whereby 
when the aurally normal user receives a communication they are able to reply through 
the microphone, transmitting a message to be transformed and displayed to the aurally 
handicapped users). 
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As to claims 13 and 27, Sakiyama et al. teach inputting verbal communication 
comprises speaking (a microphone for inputting a voice, col. 8, lines 24-26). 

As to claims 14 and 28, Sakiyama et al. teach inputting verbal communication 
and said step of responding by inputting additional verbal communication are 
undertaken by different users, (Fig. 22 depicts a communication network, where output 
is to both users through monitors, (elements 7 and 28), whereby when the aurally 
normal user receives a communication they are able to reply through the microphone, 
transmitting a message to be transformed and displayed to the aurally handicapped 
users). 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 3, 7-9, 17, and 21-23 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Sakiyama et al. 
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As to claims 3 and 17, Sakiyama et al. teach computing device is selected from 
the group consisting of personal computers, workstations, servers, clients, mini- 
computers, main-frame computers, laptop computers, mobile computers, palm-top 
computers, hand-held computers, set top boxes for a television, web-enabled 
televisions, interactive kiosks, personal digital assistants, interactive wireless 
communication devices, web-enabled wireless communication devices, mobile web 
browsers, pagers and cellular phones, (a computer with a voice interface, (col. 8, lines 
27-28). It would have been obvious to one of ordinary skill in the art at the time of the 
invention that this computer could be any type of computer, including a personal 
computer, laptop computer, mobile computer, or client computer, to increase the 
flexibility of the system). 



As to claims 7 and 21, Sakiyama et al. teach the graphic image is selected from 
the group consisting of static pictures, moving pictures, and animations (computer 
graphic (CG) patterns) (assigning a CG patterns to the corresponding sing language 
word (col. 8, lines 38-40), where it would have been obvious to one of ordinary skill in 
the art at the time of the invention for the patterns to include, static pictures, moving 
pictures, and animations as communication involving sign language involves postures 
that are static and also moving, and as a result the static pictures, moving pictures, and 
animations would be needed to properly display the language). 
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As to claims 8 and 22, Sakiyama et al. teach the speech recognition processor 
comprises a syntax module (converting the voice input into kana and kanji coexistent 
sentences, and then using these sentences to link together the corresponding sign 
language animations, and rearranging sing language words so they meet a sign 
language-like expression, (col. 8, lines 26-28, and col. 10, 39-42). It would have been 
obvious to one of ordinary skill in the art at time of the invention that as the words are 
being rearranged so that they may meet a sing language-like expression, they would be 
placed in the order that creates the best grammatical expression). 

As to claims 9 and 23, Sakiyama et al. teach the speech recognition processor 
comprises a phrase correlator, (a spoken language processor that performs analysis on 
the inputted voice language composition, finds each of the sign language words, and 
then creating sign language-like expression from the sign language words, col. 8, lines 
37-44). It would have been obvious to one of ordinary skill in the art at the time of the 
invention that every phrase or grouping of sign language words that has the same 
meaning would have the same sign language-like expression created for it to increase 
the ability of the aurally handicapped person to speak with an aurally normal person). 



Conclusion 
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5. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Goldenthal et al. (5,884,267), Waters et al. (5,657,426), Lee 
(6,014,625), and Greene, Jr. et al, (6,377,925). 

Goldenthal et al. teach aligning images to the output speech. 

Waters et al. teach producing a visual output tied to the speech output. 

Lee teaches producing a lip movement display pertaining to the output speech. 

Greene, Jr. et al. teach delivering multiple streams of data to a user, where each 
of the streams are delivered simultaneously and contain audio, visual and textual 
formats. 

6. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Thomas E. Shortledge whose telephone number is 
(571)272-7612. The examiner can normally be reached on M-F 8:00 - 4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Talivaldis Smits can be reached on (571)272-7628. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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